Acquisition of Human Faces Using a Measurement-Based Skin Reflectance Model

The scope of this thesis is threefold: design and construction of a 3D facial scanning system, development of a practical reflection model describing reflectance properties of human skin, and an analysis of facial appearance of multiple subjects, leading to intuitive editing operations for the skin model. The scanning system consists of two devices that separately acquire surface reflectance and subsurface scattering properties, respectively. Together they capture all appearance parameters of an individual face needed to generate novel, synthetic images of the face. The former device consists of sixteen cameras and 150 light-sources attached to a hemisphere surrounding the scanned subject. In order to acquire the reflectance field of the face, the light-sources are sequentially flashed, while all cameras simultaneously capture the lit face. The latter device is a contact device that uses a bundle of optical fibers to feed light into the skin of the subject, and to collect exiting radiance at different distances to the source fiber. The second integral part of the thesis has been to develop an appropriate reflectance model that allows a compact representation, as well as an analysis across multiple subjects with varying skin properties. The skin model decomposes the high-dimensional bidirectional scattering surface reflectance distribution function (BSSRDF) of skin into components that can be estimated from the measured data. The model is intuitive, amenable to interactive rendering, and easy to edit. Using the custom-built acquisition devices, a large group of people has been scanned, providing 3D face geometry, skin reflectance, and subsurface scattering for each subject. The data has been fitted to the model, and an analysis of the reflectance data across the datasets revealed variations according to subject age, skin type, gender, and external factors (heat, cold, makeup, etc.). We use two alternative instantiations of our model. In one case, we derive a low-dimensional model using non-negative matrix factorization (NMF) that spans the space of skin reflectance in our database. In the other case, histogram matching techniques are used to transfer skin characteristics between subjects. Both techniques allow a user to control skin appearance by meaningful parameters—such as skin type, gender, and age—and change the overall appearance of a person (e.g., making a Caucasian face look more Asian) or change local features (e.g., adding moles, freckles, or hair follicles). These controls make our model an intuitively editable skin reflectance representation that is suited for animation and photo-realistic rendering of human faces, preserving the natural appearance of skin.